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Sobolev Institute of Mathematics 

Abstract: In the first part of the paper we study stochastic integrals of a 
nonrandom function with respect to a nonorthogonal Hilbert noise defined on 
a semiring of subsets of an arbitrary nonempty set. 

In the second part we apply this construction to study limit behavior of 
canonical (i.e., degenerate) Von Mises statistics based on weakly dependent 
stationary observations. 



1. Stochastic integrals of non-random kernels for non-orthogonal 
noises. 

1.1. Introduction and statement of the main result. 

Let {fi, 0, P} be a probability space, X be an arbitrary nonempty set, and 971 be 
a semiring with identity of its subsets (i.e., X € 9Jl and, for all A,B £ Tl, we 
have An B G dJl and A \ B — J2i<n where Ci € 071). We call a random process 
{fi{A), A G 9Jl} an elementary stochastic measure or a noise if fJ-{A) G L2{i^, P) for 
all A G 9Jl (i.e., fi{-) is a Hilbert process) and (Nl) fj,{AiUA2) = fi{Ai) + fi{A2) a.s. 
if only Ai n ^2 = and Ai U A2 G 9Jl. 

A noise /i is called orthogonal if 
(N2) Efi{Ai)fi{A2) = mo(Ai n A2), 

where toq a- finite measure (the structure function) on a{dJl) 
Typical Examples. 

(i) Consider the following semiring of subsets of a closed interval [0, T]: 

Tt = {{t,t + 6]; <t <t + 6 <T}[j{[0,5]: 0<S<T}. 

A random process £,{t) defined on [0,T] generates the noise 

^ii{t,t + ^]) ■.= at + S)~m, 

where, in the case t = 0, this formula defines the measure of the closed interval [0, S]. 
If ^(t) is a process with independent increments then /x is an orthogonal noise. 
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(ii) To construct multiple stochastic integrals the semiring 971*^ :— dJl x ■ ■ ■ x Tl 
is considered, where 9Jl is defined above, and the following multiple noise is defined 
by increments of a random process ^(t): 

i<k 

where {t,t + S] = {ti,ti + Si] x ■ ■ ■ x {tk,tk + Sk]. 

It worth noting that, in general, in the second example the noise fi does not satisfy 
condition (N2) even if the process (,(t) has independent increments. 

We note some significant results in the area under consideration: 

I. Univariate stochastic integrals based on orthogonal noises: 

N. Wiener, 1923. 

A. N. Kolmogorov, H. Cramer, 1940. 

1. I. Gikhman and A. V. Skorokhod, 1977. 

II. Multiple stochastic integral with a multiple noise generated by a process with 

independent increments: 
N. Wiener, 1938, 1958. 
K. Ito, 1951. 
P. Major, 1981. 

III. Univariate stochastic integral with a noise generated by increments of a Hilbert 
process on the real line (nonorthogonal noise): 

M. Loeve, 1960. 

S. Cambanis and S. Huang, 1978. 
V. Pipiras and M. S. Taqqu, 2000. 

IV. Multiple stochastic integral with a multiple noise generated by increments of a 

Gaussian process on the real line {nonorthogonal noise): 

S. Cambanis and S. Huang, 1978. 

A. Dasgupta and G. Kallianpur, 1999. 

General Case. We begin to study stochastic integrals with nonorthogonal noises 
defined on semirings of subsets of an arbitrary measurable space. We follow the 
generality considered in [l3|j where general stochastic integrals with orthogonal 
noises were studied. Complete proofs of some statements in the first Section of the 
paper are published in [3]. 

Introduce the function m{A x B) := Eii{A)ii{B) indexed by elements of 9Jt^. 

Main Assumption. The function to is a finite cr-additive signed measure (cova- 
riance measure) on 9Jl^. 

Example. Let s) :~ E^(i)^(s) be the covariance function of a centered Hilbert 
random process ^(t) defined on a closed interval [0,T]. We say that the function 
<I>(t, s) possesses a bounded variation if, for a constant C, 

sup 5]|A$(t„s,)| <C, 

{*.,..} 

where A$(t,,s,) := $(ti+i,Sj+i) +^{t,,Sj) - $(ii+i,Sj) - $(ti,Sj+i) {the double 
difference); — to < ti < ■ ■ ■ < tn = T, = sq < si < ■ ■ ■ < si — T are arbitrary 
finite partitions of the interval [0,T], and the suprcmum is taken over all such 
partitions (sec also |9i). 
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Proposition 1. If ^{t, s) has a bounded variation then the Main Assumption for 
the corresponding covariance measure is valid 

It is well known that any measure of such a kind can be uniquely extended onto 
(t(9JI^). Moreover, m = to"'' — m~ (Hahn- Jordan decomposition), where to"*" and 
m~ are nonnegative finite measures. Put |to| := m+ + m~ (the total variation 
measure). 

Introduce the space of <T(97l)-measurable functions: 

S := {/ : / f{t)f{s)m{dt x ds) < oo}. 

For any cr(9Jl)-measurable functions f,g d S consider the bilinear symmetric func- 
tional 

d{f,9) / f{t)g{s)m{dtxds). 

It is clear that d{f, f) > 0. But, in general, the equation d{f, f) — has not only 
zero solution. Denote ||/|| := d{f,f)^/^ (seminorm). If S is the factor space w.r.t. 
to the condition d{f, /) = then S is an Euclidean space. But, in general, S may 
be incomplete (i.e., is not Hilbert) ([2^). 

Notice that the space S can also defined in such a way: 

S^{f. [ \f{t)f{s)\\m\{dtxds)<^}. 
Jx^ 

1/2 

But the functional ||/||* :— [J^2 \f(t)f{s)\\'m'\{dt x ds)) may not satisfy the tri- 
angle inequality (||/||* is a seminorm iff \m\ is nonnegatively defined). 

For an orthogonal noise /i with a structure function toq the following obvious 
equality chain is valid: ||/|| = ||/||, = ||/||L2(3e,mo)- 

Theorem 1. Let f G (5, || • ||). Then there exists a sequence of step functions 

(1) /„(a;) :=^Cfc/(xe Afc), 

where Ck G M, ^fc G 971, converging in (5, || • ||) to f as n ^ 00. Moreover, the 
sequence 

(2) 77(/n):=E^^-/^(^fc) 

k<~n 

mean-square converges to a limit random variable r]{f) which does not depend on 
the sequence of step functions {fn}- 

Proof. (For detail see [4]). Let {/„} be a sequence of step functions of the form (1) 
converging to / in the seminorm || • ||. One can prove that the sequences of such a 
kind exist. Then 

id 

^ll/n-Ml'-O 

as n, fc 00 due to the triangle inequality (i. e., {fn} is a Cauchy sequence in S). 
Without loss of generality, we may assume here that the step functions fk and /„ 
are defined on a common partition {A^}. Hence {??(/»)} is a Cauchy sequence in 
the Hilbert space L2{^,P). Thus, in this Hilbert space, there exists a limit random 
variable ?](/) for the sequence {ri{fn)}. □ 
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Remark 1. Since, in general, the space S is incomplete then, in this case, one 
cannot construct an isometry (one-to-one mapping preserving distances) between 
the L2-closed linear span of all integral sums and S ([22j). Existence of such isometry 
is a key argument of the classical construction of stochastic integrals with orthogonal 
noises. 

Remark 2. The generality in Theorem 1 allows us to define stochastic integrals 
both for the univariate and multivariate cases studied by the predecessors mentioned 
above. In the case of Gaussian noises generated by arbitrary centered Gaussian 
processes on the real line, our construction differs from that in Q, where multiple 
stochastic integrals are defined by the corresponding tensor power of the reproduc- 
ing Hilbert space corresponding to the above-mentioned initial Gaussian process. 
However, one can prove that, to define these multiple integrals in the Gaussian 
case, the descriptions of the corresponding kernel spaces in these two constructions 
coincide. 

Remark 3. If we consider the introduced-above multiple stochastic integral with 
the product-noise generated by a White noise with a structure function mo, and, 
moreover, the kernel vanishes on all diagonal subspaces then our construction co- 
incides with the classical Wiener - Ito multiple construction. Notice that, in this 
construction, for the kernels with zero values on all diagonal subspaces, there exists 
the isometry mentioned in Remark 1. In this case, the space S coincides with the 
Hilbert space L2 (•^'^ , m-o ) , where k is the dimension of the multiple integral. 

1.2. Infinitesimal analysis of covariance measures. 

We now describe some function kernel spaces to define the stochastic integrals in 
Theorem 1. We start with the univariate construction. 

1.2.1. Univariate stochastic integral. 

Gonsider a centered random process ^(t) with a covariance function <I>(f, s). In all 
the examples of Section I we put X — [0, T]. This process generates the elementary 
stochastic measure pL{dt) dS^{t) introduced above. 

Regular covariance functions. In the above-mentioned definition of the double 

difference of the covariance function $ we set ij+i := tj -\- 6 and Sj+i ;= Sj + S. 
Assume that, for all 5 > and tj, Sj,tj + S, Sj + 6 £ [0,T], 

A^{t„s^)=J J qit,s)X{dt)\{ds), 

where A is an arbitrary cr-finite measure. If 

/ \f{t)f{s)q{t,s)\\{dt)X{ds) <^ 

then / e S' (i.e., / f{t)d£,{t) is well defined.) 

For example, the regular FBM has the covariance function 
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where h e (1/2, 1]. In this case, q{t, s) := h{2h - l)\t - sp''-^, t ^ s, and \{dt) := 
dt is the Lebesgue measure. Moreover, in this case one can prove the embedding 
Li/h{X,dt) C S. ([H.) 

Irregular covariance functions. Consider the class of factorizing covariance 
functions 

s) = G'(min(t, s))i7(max(t, s)). 

It is known 3] that '^{t, s) of such a kind is the covariance function of a nondegen- 
erate on (0, T) random process iff the fraction is a nondecreasing positive func- 
tion. In particular, any Gaussian Markov process with non-zero covariance function 
admits such factorization: For example, a standard Wiener process has the com- 
ponents G{t) = t and H{t) = 1; a Brownian bridge on [0,1] has the components 
G{t) — t and H{t) = \ — t; and, finally, an arbitrary stationary Gaussian process 
on the positive half- line has the components G{t) = exp(ai) and H{t) = exp(— at), 
where a > 0. 

Notice that, in this case, the function s) is nondifferentiable on the diagonal 
if the components are nondegenerate functions. 

Let, in addition, G{t) "f, H{t) [ be monotone, positive on (0, T), and absolutely 
continuous w.r.t. the Lebesgue measure on [0, T]. We prove that supp ra~ = [0, T]'^\ 
D, where D := {{t,s) : t = s}, is the main diagonal of the square [0, T]^. Let 
s<s + S<t<t + A. Then 

m( (s,s + 5] X {t,t + A]) 

= G{s)H{t) + G{s + S)H{t + A)- G{s)H{t + A) - G{s + 5)H{t) 
= (G(s + 6)- Gis)){H{t + A) - Hit)) < 0. 

In other words, m~ is absolutely continuous w.r.t. the bivariate Lebesgue measure 
A2 and the corresponding Radon - Nikodym derivative is defined by the formula 

^{t,s)^G'{t)\H'{s)\. 
dX2 

We now calculate m"'"-measure of an infinitesimal diagonal square: 

m+((s, s + h] X (s, s + h]) 

= H{s + h){G{s + h)- G{s)) - G{s){H{s + h) - H{s)) 

fS+h 

{H{z)G'{z) - G{z)H'{z))dz + o{h). 

Hence m+ is absolutely continuous w.r.t. the induced Lebesgue measure on the 
diagonal. 

Therefore, in the case under consideration, the measures mT and rri^ are singu- 
lar. Finally, to verify the condition f €z S we need to verify existence of the following 
two integrals: 



/ f{t)\H{t) + l)dG{t), f }{t)\G{t) + l)dH{t). 
Jx JX 
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Covariance functions of mixed type. Let {X„; n > 1} be a stationary se- 
quence of r.v.'s satisfying mixing condition. Consider a centered Hilbert process 
(not necessarily Gaussian!) Y(t) with the covariance function which is well defined 
under some restrictions on the mixing coefficient (see the Gaussian case in [l|): 

EY{s)Y{t) = F(min(s, <)) - F{t)Fis) + ^ {F, (s, t) + F, {t, s) - 2F{s)F{t)) , 

i>i 

where F{t) is the distribution function of Xi which is assumed absolutely continuous 
with a density p(t), and Fj(t,s) is the joint distribution functions of the pairs 
{Xi,Xj+i). Let all the functions Fj{t,s), j = 1,2,..., have bounded densities 
Pj{t, s). For all t, s E R, we assume that the series 

b{t,s) -.^ J2^p,{t,s) +Pj{s,t) - 2p{t)p{s)] 
i>i 

absolutely converges and the corresponding series I ' I integrable on R^. 

We note that, under the above-mentioned restrictions, we deal with a covariance 
function represented as a sum of covariance functions from items A.l and A. 2. Hence 
we may use the infinitesimal analysis of the corresponding covariance measures from 
these items. Indeed, 

m{{t,t + A] X (s,s + A]) 

pt+A j-s+A 

= P{Xie{t,t + A]n{s,s + A]) + J J {h{u,v) ~p{u)p{v))dudv. 

So, under the conditions J p{t)p{t)dt < oo and / / \f{t)f{s)b{t,s)\dtds < oo, we 
can correctly define the stochastic integral J f{t)dY{t). 



1.2.2. Multiple stochastic integral. 

We study multiple stochastic integrals (MSI) based on a product-noise defined in 
Example (ii) by increments of a Gaussian process. In this case, to calculate the 
covariance measure we use the following well-known convenient representation: 

m((ti,ii+(5] X ••• X {t2kMk + 5]) = Y^'Y\A^{t,,t.j), 

where the sum is taken over all partitions on pairs of the set {1,2,..., 2fc}, and 
the product is taken over all pairs in a fixed such partition. Notice that, in the 
sequel, to define multiple stochastic integrals we use only this property of Gaussian 
processes. However, we may study the multiple integrals for the non-Gaussian case: 
For example, if the integrating process ^(<) can be represented as a polynomial 
transform of a Gaussian process. In this case, to define the multiple integral, we 
can obtain some restrictions on the kernel / close to the conditions below. 

Remark 4. The Main Assumption for the covariance measure m{Ai x • • • x A2k) 
introduced above follows from the Proposition 1 if only s) has a bounded 
variation. This property of the covariance function is fulfilled in items B.l - B.3 
below. 
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Regular covariance functions. Conditions to define MSI: 

/ 1/(^1, ■ • ■ ,tk)f{tk+i,. ■ ■ ,i2fe)| V'TT • ■■dt2k < oo, 

J[0,T]^>' ^-1.-1- 

where the sum and the product are introduced above, q{t, s) is the density (the 
Radon-Nikodym derivative) of A<I>. As a consequence, we obtain the main result in 
Q, for the regular FBM from item A.l. In this case we should set in this condition 
q{t,s) :=/t(2/i-l)|t-s|2'»-2^ 



Factorizing covariance functions. Let the factorizing components H and G 
be smooth functions. 

1) If ti = tj then, as (5 ^ 0, we have the following asymptotic representation of the 
double difference of <&(•) on the infinitesimal cube {ti, ti + 6] x {tj, tj + 5\ : 

A$(t„ t,) = 6{H{t,)G'{ti) - G{t,)H'{t,)) + 0((52), 

2) If t, 7^ tj then, as (5 ^ 0, 

A$(t,,tj) = (5^G"(min(tj,ij)i7'(max(i,,ij)) +o((5^). 

Denote 

51 (i) ■.= H{t)G'{t)-G{t)H'{t), 

g2{t, s) := G"(min(t, s))H' {ma.x{t, s)). 

A set -D(ri....,ri) C [0, r]^'^ is called a diagonal subspace determined by variables 
of multiplicity ri, . . . , r; (r^ > 2, "^ri < 2k) if it defines by the following I chains 
of equalities: 

where ij^„i ^ inA for any (j, m) ^ {n,d). 

Proposition 2 (see Borisov and Bystrov, 2006a). In the case under consideration 
any covariance measure m has zero mass on any diagonal subspace D(^ri,...,ri) having 
at least one multiplicity > 2. 

Given a kernel f{ti, ...,tk) we set ipf{ti ■ ■■t2k) ■= f{ti, ■ ■ .,tk)f{tk+i, ■ ■ ■,t2k)- 
Conditions to define MSI: First, we need to verify the condition 



(3) 



n 

Y]_9liSi)^Y\.^32{ti,tj)\dsi ■ --dsndti ■ ■■dt2(k-n) < 



X 

1=1 



for all n = 0, 1, . . . , fc, and, second, to verify finiteness of all the analogous integrals 



for all permutations of 2k arguments of the kernel ipf. Here, by definition, Jl ~ 1- 

i=l 

If the kernel /(•) is symmetric and vanishes on all diagonal subspaces, and the 
functions gi and g2 are bounded then condition (3) is reduced to the restriction 
/ £ L2{X'', dti ■ ■ ■ dtk). In particular, if the multiple noise is defined by increments 
of a standard Wiener process then gi = 1 and 52 = (cf. 17, 19|). 
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Covariance functions of mixed type. We now define the multiple stochastic 
integral for a Gaussian process Y{t) with the covariance introduced in A. 3. Let p{t) 
and b(t, s) defined in A. 3 be continuous functions. Then, as (5 — > 0, we have for 
ti — tj (see A. 3): 

A<!>{U,tj)^SpiU) + o{S). 

If ti ^ tj then 

A^{t,,tj) = 5^ {b{t,, t,) ~ p{t,)p{tj)) + o{5^). 

So, we actually repeat the arguments from item B.2 and to define the multiple 
stochastic integral for Y{t) we need to verify condition (3) for gi{t) := p{t) and 
g2{t,s) := b{t,s). 



2. Asymptotics of canonical von mises statistics. 

In this Section we consider some applications of the MSI construction from Section I. 
We study limit behavior of multivariate Von Mises functionals of empirical processes 
based on samples from a stationary sequence of observations. 

Let {A"„; n > 1} be a stationary sequence of [0, l]-uniformly distributed r.v.'s 
satisfying the ■0-mixing condition: ip(m) ^ if m — > oo, where 



(4) 



■0(771) := sup 



P{AB) 



P(A)P(B) 



1 



TO = 1,2, . 



and the supremum is taken over all events A and B (having non-zero probabilities) 
from the respective cr-fields JFf and T^^^, where Tf , I < k,is the cr-field generated 
by the random variables A';,...,Arfc, as well as over all natural k. This mixing 
condition was introduced in Blum, Hanson and Koopmans, 1963. 

Introduce the normalized d-variate Von Mises statistics (or V -statistics) 



(5) 



-d/2 



Ki 



_,...,id<n 



71=1,2, 



where the kernel /(•) satisfies the degeneracy condition 



E/(ii 



, tk-i,Xk, t/c+i, . . . , trf) = 



for_all ti, . . . , td G [0, 1] and k = 1, . . . , d. Such canonical statistics were introduced 
and llSl. where, moreover, the so-called U -statistics were studied: 



25| 



E 

l<'ii < ■ ■ ■ <2rf <n 



where, as a rule, the kernel /o is symmetric w.r.t. all permutations of the arguments. 

Notice that in the definitions above we may consider the observations {Xi\ taking 
values in an arbitrary measurable space and having an arbitrary distribution. If the 
sample distribution has no atoms then an [/-statistics with a symmetric kernel can 
be easily reduced to a ^-statistics with a new kernel having zero values on all diag- 
onal subspaces. For the IID samples limit behavior of these statistics is well known 
(for reference see Korolyuk and Borovskikh, 1994). In the case of one-dimensional 
observations, using the corresponding quantile transforms, we may reduce these 
statistics to those based on samples from the [0, l]-uniform distribution. 
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Introduce the normalized empirical process Sn{t) := \/n{F*{t) — t), where F*{t) 
is the standard empirical distribution function based on the above-introduced sam- 
ple X\, . . . ,Xn with the [0, 1] -uniform marginal distribution. Wc now recall the 
well-known key representation of the canonical F-statistics (5) by the Lebesgue 
integral: 

(6) Vn= j f{xi,...,Xd) dSn{xi)---dSn{Xd). 

[0,1]<* 

We recall that a diagonal subspace in [0, 1]"^ is determined as 
A*,g. := {{tu . ■ ■ ,td) e [0,1]'^: 

til ~ ti2 ' ' ' ~ ^iq-^ ■>■■■■> + i ~ tigr_i+2 

where i* := (ii, Z2, . . . , , . . . , iq^_^+i,iq^_-^+2, . . . ,iq^) is a vector with pairwise 
different integer coordinates from the set {1, . . . , d} and q* := {qi, . . . ,qr), where 
Qi > Qr < d and qi+i — > 2 if 1 < j < r. In the sequel, to indicate diag- 
onal subspaces, we will use natural parameter i instead of vector- valued {i*,q*) 
renumbering all the diagonal subspaces. 

The subspace of [0, 1]'' defined by pairwise different coordinates U is called the 
main subspace and denoted by Dq. Obviously, 

Do=[0,l]''\\jDi. 

i>i 

Introduce the following function space 

So~{f:^f f{h, . . .,td)dti^^ ■ ■ ■ dti^^ < oo}, 

where the subscripts iq^, . . . , iq^ determine the corresponding diagonal (or the main) 
subspace Di of dimension r and the kernel f^(-) in the corresponding multiple 
integral of this notation has only r independent arguments. For example, if 
A = {(ii,i2,t3,t4) e [0,1]^ : ti=t3, t2=t4} then 

/ f^{tl,t2,t3,t4) dt3dt4 = / P{t3,t4,t3,t4) dt3dt4. 
JDi J[0,l]2 

Notice that, under the conditions of Theorem 2 below, the function space 5*0 is 
a subspace of S introduced in Theorem 1 for d-fold product-noise generated by 
increments of the stochastic processes Y{t) from items A. 3 and B.3 since the series 

h{t, s) = Y^ {pk {t, s) +Pk{s,t)- 2) 

k>l 

from item A. 3 satisfies the restrictions we need and, moreover, it is uniformly 
bounded on [0, 1]^ due to ^-mixing condition (4) and the restrictions on the mix- 
ing coefficient in Theorem 2 below. First of all, wc observe that (4) implies exis- 
tence of joint densities Pk{t,s) since we may consider in (4) 'infinitesimal events" 
A := {Xi e {t,t + dt)} and B := {Xk+i G {s,s + ds)} and take the relations 
P{A) = dt and P{B) = ds into account. Moreover, we obtain the upper bound 
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s)\ < J2k>i V'(^)- This fact allow us to estimate the covariance measure of the 
corresponding product-noise generated by increments of Y{t). 
Define on the space the combined L2-norm 

(7) ll/llo:=E / fiti,.-.,td)dU,,---dU^^, 

where in the case i = {i. e., for the main subspace Dq,) we put = k and r = d 

in the corresponding summand on the right-hand side of (7). Notice that this norm 
is stronger than 1 1 • 1 1 since in the case under consideration we have gi (t) := p{t) = 1 
on [0, 1] and g^it, s) := b{t, s) is a bounded function as well (see B.3 and the upper 
bounds in (20) and (21) below). In other words, in the case imdcr considerations, 
the linear normed space (^o, 1 1 • | |o) is embedded to the corresponding space (5, 1 1 • 1 1) 
in Theorem 1. 

Theorem 2. Under the above-mentioned restrictions let f G So and 

(8) ^{d) ■.= J2i'ik)k^'^~^ <oo. 

k>l 

Then, as n —> oo, 

Vn^ [ f{t,,...,td)dY{ti)---dY{td), 
>/[0,l]<* 

where Y{t) is a centered Gaussian process with the covariance 

EY{t)Y{s) = mm{t, s)-ts + Y^ {Fk{t, s) + Fk{s, t) - 2ts) . 

fe>i 

Remark 5. In the IID case {'ip{-) = 0, 6(-, •) = 0, and Fk{t, s) = ts) the conditions 
of Theorem 2 coincide with the traditional well-known restrictions in Mises, 1947, 
Filippova, 1959, 1962, Borisov and Sakhanenko, 2000, and others. In this case, the 
process Y{t) = W'^{t) is a Brownian bridge. Notice also that, in the IID case, the 
multiple stochastic integral w.r.t the product-noise generated by W'^{t} in Theorem 
2 coincides in distribution with the analogous integral w.r.t. a White noise (gener- 
ated by a standard Wiener process W{t)) due to degeneracy of the kernel and the 

well-known representation W^{t) = W{t) — tW{l). 

If a kernel /(ti, . . . ,td) vanishes on all diagonal subspaccs then sufficient con- 
ditions for / to be an element of So is as follows: / G i2([0, 1]'', dt\ - ■ ■ dtd)- The 
kernels of such a kind are used to describe limit behavior of canonical f-statistics 
(for example, sec Major, 1994, and the corresponding comment above). Moreover, 
to describe the Ito - Wiener construction of multiple integrals the kernels vanish- 
ing on all diagonal subspaces provide the above-mentioned isometry between closed 
linear span of multiple integral sums and the corresponding function space. 

Remark 6. From the beginning, in the IID case, to describe limit behavior of 
canonical U and ^-statistics another representation of the limit distribution was 
used. For instance, in the case d = 2, for a canonical ?7-statistics with a symmetric 
kernel the limit random variable can be represented as the random series 

oo 

(9) ^Afe(Tl-l), 

fe=l 
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where {t^} are i.i.d. standard Gaussian random variables, j Afcl are eigenvalues of 
the integral operator in i2[0, 1] with the kernel_.f(s, i) (see [25|). Later this result 
was extended to the general case d > 2 (see [2j]). In the general case the limit 
random variable can be represented as a polynomial transform of {t^} in terms 
of the Hermite-Chebyshev polynomials. Finally, using analogous representations of 
multiple Wiener integrals noted in [27] , and in [17| , one can rewrite the limit random 
variables as the corresponding multiple stochastic integral w.r.t. to a multiple White 
noise (see, for example, Dynkin and Mandelbaum, 1983). 



In the non-IID case an analog of (9) was obtained in pj|, where {Afe} are the 
above-mentioned eigenvalues which are additionally assumed to be summable, and 
{rfc} is a Gaussian sequence with the covariance function depending of these eigen- 
values as well as of the covariance function of the initial stationary sequence {Xi} 
satisfying 93-mixing condition. 

In the special case when the observations are defined by a nonrandom transform 
of a Gaussian stationary sequence under another dependency restriction, limit be- 
havior of canonical U -statistics were investigated in (sj . 

Proof of Theorem 2. We divide the proof into three steps (for detail see [H| ) . Let 
{fkixi, . . . , Xd)} be a sequence of step functions of the form 

d 

(10) fk{xi,...,xd) := fn.■■■,uY[Hx^ ^n) 

converging in (Sq, || • ||o) to a function f{xi, . . . ,Xd) as k ^ 00. Hence the functions 
{fk} converge to / in the corresponding linear normed space (iS, 1 1 • 1 1) in Theorem 1. 
Then 

S t e p I. For every fixed fc, due to the multivariate central limit theorem for 
finite-dimensional distributions of the empirical measure based on slowly dependent 
observations (see, for example, [l[), as n — > cxd, 

fkixi,...,Xd)dS„{xi)---dSn{xd) / fkixi,...,Xd)dY{xi)---dY{xd)- 



'[0,1]'' "'[0,1]'* 

Actually, this weak convergence is valid under weaker dependency conditions than 
those in Theorem 2. 

S t e p II. As fc — > 00, by the above-mentioned construction of multiple stochastic 
integrals (see Theorem 1 and the comment above) we have the following mean 
square convergence: 

fk{xi,...,Xd)dY{xi)---dY{xd)^ / f{xi,...,Xd)dY{xi)---dY{xd). 

Step III. As fc — > 00, we should prove the following mean square convergence 
uniformly on n: 

fk{xi, . . . ,Xd)dSnixi) ■ ■ ■ dSn{Xd) ^ / f {xi , . . . , Xd)dSnixi) ■ ■ ■ dSn{Xd) ■ 

[OS]'' J[os]'' 

It is easy to see that the assertion of Theorem 2 follows from items I - III. 
So, we should prove only item III. To establish the last convergence, for every 
step function gk of the form (10), it suffices to prove the estimate 

(11) supE / gk{xi,...,Xd)dSn{xi)---dSn{xd)] < C||5fe||o, 
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where the constant C depends on d and '^{d) only. Indeed, this estimate impHes 
the analogous upper bound for the step function fk — fi (see the corresponding 
comment in the proof of Theorem 1), where fk and /; are arbitrary elements 
of the above-mentioned sequence {/m} converging to the kernel / in the norm 
II • llo- Therefore, as I — > oo, the corresponding L2-limit of the integral sums 
/ fi{xi, . . . , Xd)dSnixi) ■ ■ ■ dSn(xd) exists and coincides with the Lebesgue integral 
in (6) since this limit does not depend on a sequence of step functions approximating 
to /. Thus, as /c ^ oo. 



supE (^J fk{xi, ■ ■ ■ ,Xd)dSn{xi) ■ --dSnixd) 



f{xi, . . . , Xd)dSn{xi) ■ ■ ■ dSn{xd)j -> 

for every sequence of step functions {fk} converging to the kernel / in (Sq, \ \ ■ ||o) 
if only relation (11) is valid. 

To prove (11) we need the following auxiliary statements. 

Lemma 1. (see Let ^ and r/ be random variables measurable w.r. t. the T\ 

and [rn > 1) respectively. If E\£,\ < oo and E\r]\ < oo then 

(12) \ECv-E^ETj\<^{m)mm. 

Introduce the following notation: Ik{A) := I{Xk £ A) — P{A), A G A, where 
the marginal P is the Lebesgue measure on [0, 1]. The following assertion is easy 
proved by induction (for detail see [5]). 

Lemma 2. For any natural numbers q and li, . . . , Iq, and for any pairwise disjoint 
measurable subsets Ai, . . . , Aq the following moment inequality is valid: 

(13) E\?,^ (A,) {Aq)\ <{q + l)F(Ai) . . . P{Aq). 

From Lemmas 1 and 2 we deduce the following upper bound. 

Lemma 3. Let qi < ■ ■ ■ < qs be arbitrary natural numbers and qo = 0. Consider 
s collections of sets {Ai, . . . Aq^}, . . . , {Aq^-^^i, . . . Aq^ }, where the measurable sub- 
sets Ai inside of every collection are pairwise disjoint. Put 

^fc. :=5:'-+^(A,._,+i)---4'(Aj- 

Then, for any natural numbers fci < • • • < fcs and li, . . . ,lq-^, . . . ,lq^, the following 
estimate is valid: 

(14) Eli^k, ■ ■ - i^kA < C{i^{l),s,qs)P{A,) ■ ■ ■ P{AqJ, 

where the constant C(-) depends only on the arguments indicated. 

Proof. By (13) we have E|i/fcJ < [q, - qi_i + l)P{Aq.^^+i) ■ • •P(A,J. It is clear 
that the random variables {vki} satisfy '(/'-mixing condition. Hence, by (12) and 
(13) we obtain 

E|z.fei ■■■VkA<X{{l + ^{kj+i - kj))^\vk, I • • • ^Vk^ I 

s-l s-1 
< [](1 + VXfc, + l - k,)) \{{q, - + l)F(Ai) • • • P(A,J 

<C(^(l),s,g,)F(Ai)---P(AJ. 
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The Lemma is proved. □ 

The main assertion to prove (11) is as follows: 

Lemma 4. Let li, . . . ,lq be natural numbers such that h + ■ ■ ■ + Iq = 2d, and let 
Ai, . . . ,Aq be a collection of pairwise disjoint measurable sets. Under condition (8) 
the following upper bound is valid: 

(15) |E5iK^i) • • • Sl^{Aq)\ < CP{A,) . . . P{Aq). 

where the constant C depends only on d and ^(d). 

Remark 7. The statement of Lemma 4 is also contained in Sen, 1972. However, 
the corresponding constant in this paper contains the factor (1 + ip{0))'^. It is easy 
to see that if the marginal distribution has a continuous component (for example, 
if it is the Lebesgue measure on [0, 1]) or infinitely many atoms then = oo. To 
verify this property we may put in (4) A = B, where A is an event from the cr- field 
J^l- So, under the above-mentioned restrictions on the marginal P 

V'(0)> sup (1/P(A) - 1) = oo. 

A£y=-l, P{A)>0 

In the sequel we will denote by the symbols C or Cj various positive constants 
depending only on d and 4'(d). 

Proof of Lemma 4. We first write the simple estimate 

\ESl^{A,)---Sl^{Aq)\ 

<n-'' Yl |E4,(Ai)---4,^(Ai)...4,,_,^^,(A,)...4,,(A,)|. 

fei,...,fe2<i<n 

The initial sum on the right-hand side of this inequality can be estimated by a finite 
sum of the following diagonal subsums 

(16) Yl |Ez^fei---!^fcJ, 

where := II^^^\Ai) ■ ■ ■ ll''^''\Aq), and the integers Sj{i) are defined by the 
corresponding diagonal subspace of subscripts in the initial multiple sum and satisfy 
the conditions < Sj{i) < Ij for alH < r and j < q, and X]j<r ^j<q = ^d. 

Let r < d. Estimating by (14) each summand in (16) and taking the normalized 
factor n~'^ and the number of summands in (16) into account we obtain the upper 
bound we need. 

Let now r > d. We call the random variable Vki short product if X^j<g Sj{i) = 1, 

i. e., ffe. = Ii-i{Aq.) for some qi < q. Notice that if is a short product then 
Ez/fe. = 0. 

We now consider the auxiliary multiple sum consisting of the random variables 
defined in (16) for the fixed diagonal subspace of subscripts: 

(17) |Ei^fe.i---^fe.J' 

where 1 < Vi < V2 < r and the value v := V2 — Vi + 1 is the dimension of the cor- 
responding multiple sum. Introduce the following notation: ej{i) := min{l, Sj{i)}. 
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We first prove the following assertion: If, in the summands in (17), there are at 
least m shorts products, where < m < w, then the following upper bound is valid: 

(18) Yl lE^'^^i • • • ^-^"^ I < Cn"— /2 H P{Ajr^^^^'^^\ 

fct)i<---<fc«)2<" 3<<1 
V2 

where aj{vi,V2) ■= J2 Notice that the set function aj{a,b) is additive on 

i=vi 

intervals [a,b]. We prove this assertion by induction on m for all vi and V2 such 
that V > m and v < r. Let m = 1, i. e., the expectations in (17) contain at least one 
short product. Denote it by Vk, , where ky^ < ki < k^^ . First we note that, in terms 
of the notation above, wc can rewrite the statement of Lemma 3 for the absolute 
moment of each random product in (17) in such a way: 

Taking this estimate into account we evaluate by (12) every summand in (17) setting 
£. ■= Vk^^ ■■■Vki and r/ := Vki+i ■ ■ ■ Vk„^ : 

|E!/fe„^ • • • Vk^^ I 

< ^{ki+i-ki)E\uk,^---Uk,muk,+.,---Uk,J 

k-oi <---<ki<n ki^i<---<kv2 

< Cm"-! Yl P{Aj)"'i^''^'''^'^Y^(^ 

j<q i>l 

+ (72n''^-']JP(^,)"^('+i'"=) 

X Y V'(^;-fc(-i)E|i^fei---i^fc,_jE|i/feJ 

< (Csn^-i + dn^'-^n^-'"^) JJ P{Aj)"'i^''^''"^^ 

< C5n''-i/2 J-j- p(^^.)". (''i-''2) 

j<<! 

which required. In this chain of relations the second inequality is valid due to (12) 
and the equality Ez/fc, as well. 

We now assume that the upper bounds 

arc true for all integers z < m,, where z is the minimal possible number of short 
products in the expectations under consideration, and for all possible dimensions 
V : z < V < r oi multiple sums of the form (17), and, moreover, the moments in 
(17) contain at least m shorts products. Denote these products by Vki^ , • • . , i^ki^ ■ 
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Consider the following m — 1 pairs of neighboring products: Vk,^ '^hs+i ' s = 1 . . .m— 
1. Denote by ti, . . . , tm-i differences between the subscripts in these pairs. We have 

^ \Ei^k^^ ■ ■ ■ i^k^J < Ri + ■ ■ ■ + Rm-l, 

kvi <---<ky2<n 

where the subsum Rg is taken over the set of subscripts 

Is '•— {(^ui) • * • 5 ^vg) * ^ ' ' ' ^ ky^ ^ ^5 is ~ maxt^}. 
We now estimate by (12) each summand in Rg setting ^ := i/^^^ • • • I'k,^ and 

k-ui<-<ki^<n ki^ + i<---<k„2<n 

Consider the first sum on the right-hand side of (19): 

Yl ^iK+i - K)^Vk^^ ■ ■ ■ Vki, lEliz/c^+i • • • i//c„, I 
I, 

i<q Is 
<Cj\P{AjY^^'"^'''-'^rf-^"'-^^ limits) 

3<q ti:ti<ts 

< C]JP(Aj)"^("i'''=)n''-™/2^' . 

Notice that the last inequality is valid for m > 2. Consider now the product of the 
sums on the right-hand side of (19). Let the summands of the first sum contain 
mi short products indicated above, and, in the summands of the second sum, 
there arc to — mi short products indicated above. By the construction we have 
1 < mi < m — 1. Hence, for these sums we can use the induction proposition. 
Finally, we have 

kvi<---<ki^<n fej3 + i<---</c„2 <n 

< (7^'=-"»i/2^''-'a-(TO-mi)/2 JJ p^^j^.-^ajivi^v^) 

j<Q 

which required. Thus, for Rg we obtained the upper bound we need. It means that 
the analogous estimate is valid for whole sum (17). The induction is over. 

To prove the assertion of Lemma 4 we should note that, first, by the definition, 
Cfji^ji') > 1 for all j ^ </) and, second, in the case r > d, the summands in (16) 
contain at least 2(r — d) short products. So, we should put in (18) vi := 1, V2 ■= r, 
m := 2(r — d) and v := r. It means that, for the sum in (16), the following upper 
bound is valid: 

J2 l^'^k, ■ ■ ■ i^kj < Cn^'llPiA.r^^''-^ < Cn^'PiA,) ■ ■ ■ P{Ag). 

ki<---<kr<n j<q 
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The Lemma is proved. □ 

In conclusion we deduce from Lemma 4 the upper bound (11). For every step 
function of the form (10) we have 

supE / gk{xi,...,Xd)dSn{xi)---dS„{xd)\ 



(20) 



supE / gk{xi,...Xd)gk{xd+i,.-.X2d)dSn{xi)---dS„{x2d) 

[0,1]^'* 



< sup ^ I 

gii,...,id I \gid+l,---,i2d 

|E|5„(A,J...5„(A,,J| 

n 

il,---,i2d 

-^'Yl / \Skixi,...Xd)\\gkixd+i,...X2d)\P{dx^ )■■■P{dx^^^), 
i>0 •' 

where D^^'' is a diagonal (or the main) subspace in [0, 1]^'' of dimension r which 
is defined by the integers qi, . . . , by analogy with Di in [0, l]"*. Further, by the 
Cauchy - Bunyakovskii inequality we estimate every multiple integral on the right- 
hand side of (20): 

' , ^ \gk{xi, . ..Xd)\\gk{xd+i, ■ ■■X2d)\Pidx,^J- ■■Pidx.^J 



< 



(21) 



2 

y 

D 



glixi,... Xd)P{dxi^^ )■■■ P{dx^^^ ) 



\ 1/2 

X / gl{xi,...Xd)P{dx^,^)---P{dx^,^)\ 

j 

2 



<lk-llo, 

where Dj^ and Djr^ are the corresponding diagonal (or the main) subspaces in [0, l]"* 
of respective dimensions / and m, where Z + m > r, defined by the integers {wj} 
and {sj}. Thus, the upper bound in (11) is proved. □ 
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